Here we go again. I once said that, in the wake of study after study that fails to find activity of various “complementary and alternative medicine” (CAM) beyond that of placebo, CAM advocates are now in the midst of a “rebranding” campaign in which CAM is said to work through the “power of placebo.” Personally, I’ve argued that in reality this new focus on placebo effects as the “mechanism” through which CAM “works” is in reality more a manifestation of the common fantasy that wishing makes it so.
None of this, of course, can stop everybody’s favorite apologist for “complementary and alternative medicine” (CAM) and, in particular, using placebo effects therapeutically, from continuing to do what he does with a study that’s been widely reported in the news and even featured on Science Friday last week. Basically, it’s a study in Science Translational Medicine, in which our old friend Ted Kaptchuk teamed up with an investigator interested in migraines, Rami Burstein, to do a study that finds that believing a medicine will work can have a strong effect on its actual activity on migraine. As is the case with most studies in which Kaptchuk is involved, it’s mildly interesting from a scientific standpoint. Unlike most studies in which he is involved, Kaptchuk seems a bit more able to tone down the hyperbole, which is a good thing. Unfortunately, this study, as much as it’s being touted by the press as providing new information on placebo effects, really doesn’t tell us much that is new.
But first, in case you don’t remember who Ted Kaptchuk is, let’s take a moment to remind you, given that it’s been a while since he’s appeared as a topic on this blog. He’s the Director of the Program in Placebo Studies, Beth Israel Deaconess Medical Center, and a professor of medicine at the Harvard Medical School. His work on placebo effects has been a frequent topic right here on this very blog and has been a mixed bag. On the one hand, Kaptchuk sometimes does interesting work, but on the other hand he can’t seem to help himself when it comes to overselling it. For example, two years ago, Kaptchuk’s group published a study in which they evaluated subjective placebo effects and objective physiologic effects of “sham acupuncture” in asthma patients. The observations were actually intriguing, as I pointed out. Basically, Kaptchuk compared asthma patients receiving “placebo acupuncture” with patients receiving a real albuterol inhaler. What he found was that placebo effects from the sham acupuncture could make patients feel as though they were less short of breath, even though pulmonary function tests revealed that their lung function had not improved, a result that was not unexpected. It was also, as Peter Lipson described, a finding that indicated how dangerous it could be to rely on placebo effects to treat asthma in that it could easily result in the death of your patients by lulling them into a false sense of security of not feeling short of breath when, from a physiologic standpoint, they are on the knife’s edge of respiratory failure. Meanwhile, advocates of using placebo effects intentionally in medicine spun this study as some great evidence that placeboes could be useful in medicine when in fact it suggested that relying on placebo effects to alter physiology could be very dangerous.
The other message that Kaptchuk has been promoting is that it is possible to have “placeboes without deception.” One of the greatest difficulties as a physician with intentionally utilizing placebo effects, if they are useful, is that under our current understanding of how placeboes work deception is necessary. The patient has to be convinced that the placebo they are getting will help them, which requires the physician or health care provider, in essence, to lie. Indeed, Kaptchuk did a study a few years back testing placeboes in irritable bowel syndrome (IBS) in which he concluded that one could have placebo effects without deception. As I and others pointed out, his study showed nothing of the sort, as the power of suggestion, in which placebo pills were described as being capable of producing “powerful mind-body effects”, was used. Yet Kaptchuk’s old spin continues to persist, even in an NPR story about Kaptchuk and Burstein’s “hot off the presses” migraine study:
The group has shown: that placebos rival the effect of active medication in patients with asthma; that even when patients know they’re taking a placebo, they can get relief from the cramps, bloating and diarrhea of irritable bowel syndrome; and that those subliminal suggestions can activate patients’ placebo response.
Placeboes had no physiologic effect in the asthma study, and, I forgot to mention, in the IBS study the effects observed were actually very small and were not evidence of “placeboes without deception.”
So what about the study itself? Actually, like the asthma study, it’s a pretty well-designed study. Unlike the asthma study, whose results were often exaggerated and misrepresented to mean that placebo effects were as effective as real asthma medicine, Kaptchuk appears not to be willfully misinterpreting it, although he can’t always resist letting some of his old exaggerations slip into his discussion and interviews, as you will see. First, however, let’s look at the study.
Basically, it examined the effect of placebo or an active drug against migraine, Maxalt (rizatriptan), on the migraine headaches of 66 subjects. Each participant was asked to document seven migraine attacks, starting with one untreated attack at the beginning of the study and six subsequent attacks. These six attacks were randomly assigned to be treated with 10 mg of Maxalt or placebo, each labeled either as “placebo,” “Maxalt,” or “Maxalt or placebo.” Patients were asked to record one pain score 30 minutes after onset of the headache as a baseline and then to take the study pill, after which they were to record a second pain score 2.5 hours after the onset of the headache. They were provided with “rescue medications,” which the participants could use as needed at the 2.5 hour time point. Basically, if the participants’ headache hadn’t been adequately relieved, they could take the medication.
One of the strengths of this study is that there really wasn’t much interaction with the physicians running the study, thus minimizing the effects of personal interaction with health care providers after the first visit when subjects were recruited for the study. There ended up being six groups divided into three groups of two, as shown in the image below:
The idea is that there were three conditions: “negative information” (placebo labeling), “neutral” or uncertain information (label says that the pill could be Maxalt or placebo), and “positive information” (Maxalt labeling). Each of these conditions is either true or not; i.e., the “placebo” envelope can contain placebo or the actual Maxalt or the “Maxalt” envelope could contain placebo or the actual Maxalt.” Two main outcomes were measured: (1) the decrease in pain score from 30 minutes after onset to the 2.5 hour mark; i.e., two hours after taking the drug or placebo; and (2) whether or not the subject was pain-free at the 2.5 hour mark.
However, contrary to a lot of the discussion that occurs later expressing how “powerful” placebo effects alone were even under “truthful” conditions, there was a bit of priming going on in the study materials, as published in the supplemental data and information:
Scripted Information Read to Participants. “You are invited to take part in a research study for the purpose of understanding the effects of repeated administration of Maxalt for the treatment of acute migraine attacks, and why placebo rates are so high in migraine therapy. Our first goal is to understand why Maxalt makes you pain-free in one attack but not in another. Our second goal is to understand why placebo pills can also make you pain-free. Our third goal is to understand why Maxalt works differently when given in double-blind study vs. real-life experience when you take it at home. These goals are scientifically important for developing new therapies for migraine.
I repeat for emphasis: “Our second goal is to understand why placebo pills can also make you pain-free.” Not to understand why placebo pills might be able to make you pain-free or could possibly make you pain free. “Can make you pain free.” To be fair, this isn’t nearly as blatant as the IBS study in which subjects were told that placeboes could produce “powerful mind-body effects” in the study information. Also, mentioning that “placebo rates are so high in migraine therapy” primes the subjects to expect placeboes to work.
With this suggestion, it’s not entirely surprising that placebo effects were fairly robust. The results can basically be summarized thusly as changes in pain scores:
- No treatment: 15 percent increase in pain.
- Known placebo: 26 percent decrease.
- Placebo labeled Maxalt: 25 percent decrease.*
- Maxalt labeled as placebo: 36 percent decrease.*
- Mystery pill (Maxalt or placebo): 40 percent decrease.
- Known Maxalt: 40 percent decrease.
Note that there was no statistically significant difference between the placebo labeled as Maxalt and the Maxalt labeled as placebo. This is perhaps the most interesting finding, and suggests that positive labeling can boost placebo effects (again, not a new finding) and that negative labeling can decrease whatever contribution there is by placebo effects to the action of the real drug). As for the other values in other groups, reported differences are a lot less impressive if you look at the graph, complete with error bars:
Note the huge overlap between decreases in pain scores in the placebo group regardless of whether the label was “placebo,” “unspecified,” or “positive.” The same is true for decreases in pain scores in the Maxalt group regardless of label. This raises the question of whether reported differences, albeit statistically significantly different, are in any way clinically significant.
The second endpoint examined was whether or not the subject was pain free after 2.5 hours. It’s here where the real differences dwell:
Unlike the primary endpoint, the proportion of participants who were pain-free during the no-treatment condition (0.7%) was not statistically different from when participants took open-label placebo (5.7%). As with the primary endpoint, the proportion of participants pain-free after treatment was not statistically different between Maxalt treatment mislabeled as placebo (14.6%) and placebo treatment mislabeled as Maxalt (7.7%). The resulting therapeutic gain (that is, drug-placebo difference) was 8.8 percentage points under “placebo” labeling [odds ratio (OR), 2.80], 26.6 percentage points under “Maxalt or placebo” labeling (OR, 7.19), and 24.6 percentage points under “Maxalt” labeling (OR, 5.70).
One critical finding here is that Maxalt beat any sort of placebo effect, and not by a little bit, either. For all the Maxalt groups, the percentage of subjects who were pain free was 25.5% compared to 6.7% for all the placebo groups. That’s nearly a four-fold difference. Also note that the no treatment condition was not statistically different from the open-label placebo condition.
The error bars, however, remain wide:
So what does this all mean? In the discussion, Burstein and Kaptchuk try to sell the reader on some old ideas about placebo and some CAM-friendly ideas about placebo:
By manipulating the information provided to patients, our primary analysis showed that the magnitude of headache relief induced by Maxalt (10-mg rizatriptan), as well as that of placebo, was lowest when pills were labeled as placebo, and higher when pills had uncertain labeling or were labeled as active medication. Two other findings were that (i) placebo treatment mislabeled as 10-mg Maxalt reduced headache severity as effectively as did Maxalt mislabeled as placebo, and (ii) open-label placebo treatment was superior to no treatment. We conclude that raising the likelihood of receiving active treatment for pain relief significantly contributed to increased success rate of triptan therapy for migraine, that open-label placebo treatment may have an important therapeutic benefit, and that placebo and medication effects can be modulated by expectancies.
Although Maxalt was superior to placebo under each type of information, we were surprised that the efficacy of Maxalt mislabeled as placebo was not significantly better than the efficacy of placebo mislabeled as Maxalt. We were also surprised to find that open-label placebo treatment induced pain relief as compared with the worsening of pain during the untreated attack. A therapeutic benefit of open-label placebo versus no treatment was also recently reported for patients with irritable bowel syndrome in a randomized controlled study (8) and in a pilot study in depression (9).
Methinks the authors doth protest too much surprise at seeing placebo effects in the open label placebo group, given that the study materials suggested that subjects would experience pain relief or even be pain free based on placebo effects and subjects were told that migraines are subject to placebo effects. Mentioning Kaptchuk’s previous IBS study in which he tried to argue that placeboes without deception were possible is just another way of insinuating the idea that it’s possible to have placebo effects without deception into this study, even though superficially the authors appear to be much more straightforward about discussing their results. On the other hand, when it comes to the endpoint that people with migraines really care about, being pain-free, open label placebo was no different than no treatment at all, a point that gets lost in all the discussions of “the power of positive thinking.”
It’s also rather frustrating that the authors state that many placebo researchers would have considered the providing of information about whether or not patients were receiving placebo, real drug, or had a 50:50 chance of receiving real drug as something that would affect expectancy; i.e., the expectation of benefit. However, they didn’t make any effort to assess expectancy because they were afraid of causing patients to question the accuracy of the information provided on the envelopes. The lack of assessment of expectancy greatly decreases the utility of this study and the ability to generalize from it a potential mechanism to explain their results. Worse, no assessment of blinding was performed because the investigators were worried that this would provoke suspicions in an in-study design. Quite frankly, this is not a convincing excuse. Assessment of blinding is such a routine and key part of randomized clinical trials that to not include it in the trial design is bound to leave an opening for suspicion that the subjects might have been able to guess whether the envelope containing the pill they were using for each migraine incident contained real drug or placebo.
In the end, however, this study really doesn’t tell us much that is new or that we didn’t already know. For a subjective finding, a significant part of the drug effect appears to be placebo. We knew that, which is why I found it rather odd that Kaptchuk exulted over how “half the effect” of Maxalt is due to placebo effects. I also find it rather odd how, in his interview on Science Friday, Kaptchuk emphasized how subjects were given different expectations; yet he didn’t actually assess expectancy. He also goes on about how most studies don’t include a no-treatment control, as if that were a major observation. Of course, the reason that most studies examining subjective outcomes don’t include a no-treatment control anymore is because scientists know from previous studies that placebo effects can be significant confounders.
So the observation that there was a difference between the no-treatment control the placebo arm was not unexpected for the main endpoint, decrease in pain, because that’s a subjective endpoint. Even less unexpected is the observation that there was no statistically significant difference in the chances of being pain-free at the 2.5 hour time point between the no-treatment control and the open-label placebo group. This is entirely consistent with what we’ve been arguing here at SBM for a long time, that the more “objective” or “hard” the endpoint (and, although there is still a subjective component, being pain-free is a harder endpoint than stating a pain score), the weaker any placebo effects observed are, to the point that the very “hardest” endpoints, such as tumor regression or survival, are not affected by placebo. None of this stops Kaptchuk from emphasizing the decrease in pain scores in the open label placebo group in his Science Friday interview but not mentioning that no more people were pain free in that group than in the no treatment group.
I think we get a glimpse into Kaptchuk’s mind in the part of the Science Friday interview when he says:
I think that in the same way a physician has to calculate what pharmaceutical I have to give, how many milligrams he has to give of that drug, he or she might have to calculate the exact right words to accompany the pharmaceutical. In this study the words actually double the effect or cuts the effect of the drug in half. What exactly those words are, I think that’s more research. Our experiment was a proof of concept to see if words work. Then we now have to figure out what’s an ethical way to provide a positive message that’s true, accurate, and is not an exaggeration.
Basically, Kaptchuk appears to be saying that we have to find the right magical words to invoke the mystical placebo effect. Seriously. This experiment had only three sets of words, the two of which that mattered were either affirming that drug was present or stating that placebo was present. It’s not fancy.
Certainly, Kaptchuk does nothing to discourage headlines like:
- Migraines can be tamed with positive thinking, study finds
- Study: Thinking Positive Helps Migraine Drug Work
- Positive thinking a big factor in effectiveness of migraine pills
- Migraine Study Placebo Effect: New Study Shows The Power of Positive Thinking Makes Migraine Medicine Work
- The placebo effect: A new study underscores its remarkable power
All of which miss the point. It’s not “positive thinking.” It might be expectancy, but we don’t know that because it wasn’t assessed in this study. The two are different.
In the end, this isn’t really a bad study. It’s just that, even though he’s doing a lot better than he used to, Kaptchuk still can’t seem to resist reading a bit too much into it. He’s not as blatant about it as he used to be, but he’s still implying that placebo effects without deception are possible and desirable. It really doesn’t tell us much that we don’t already know. Placebo effects can enhance subjective effects of pharmaceuticals, expectancy can affect the perception of how well drugs work on subjective outcomes. Understanding placebo effects is important. Overselling them does not help our understanding.